cuda.core: make Device methods use their bound context by Andy-Jost · Pull Request #2750 · NVIDIA/cuda-python

Andy-Jost · 2026-09-01T20:46:48Z

Description

Device methods that create resources or synchronize (create_stream, create_event, create_opaque_array, create_mipmapped_array, create_texture_object, create_surface_object, sync) previously operated on whatever context was current on the calling thread, so dev1.create_stream() could silently create a stream on device 0 while labeling it device_id == 1.

These methods now run against the Device's bound context and restore the caller's current context afterward, including when no context is current. Details:

Stream, event, array, mipmapped-array, texture, and surface creation take the target context explicitly; the C++ handle layer switches context around the driver call and undoes the creation if the caller's context cannot be restored.
Device.sync() synchronizes the bound context. Device.set_current() returns the previously current context.
Texture and surface creation reject resources that belong to a different device or context.
Context-sensitive cleanup (default-stream frees, texture/surface destroys) runs in the owning context and warns instead of raising; handle-based destroys are left alone since they resolve their own context.
_SynchronousMemoryResource binds its context at construction.
Release notes for 1.2.0 and the interoperability docs describe the new behavior.

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

🤖 Generated with Claude Code

copy-pr-bot · 2026-09-01T20:46:51Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Andy-Jost · 2026-09-01T20:51:15Z

/ok to test

github-actions · 2026-09-01T21:08:15Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-2750/
https://nvidia.github.io/cuda-python/pr-preview/pr-2750/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-2750/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-2750/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

Run context-sensitive Device operations against the Device's bound context while preserving caller state. Centralize context-aware cleanup and synchronous allocation handling so resource lifetimes remain correct.

Andy-Jost · 2026-09-01T23:04:17Z

/ok to test

Andy-Jost · 2026-09-01T22:42:48Z

+// Run a creation operation and undo it if context restoration fails.
+// Context-independent undo always runs. Context-sensitive undo runs only
+// after verifying that the target context remains current; otherwise the
+// resource leaks rather than risking cleanup in the wrong context.
+template <typename Fn, typename Undo>
+CUresult invoke_in_context_or_undo(const ContextHandle& h_context, Fn&& operation,
+                                   Undo&& undo, bool undo_requires_target_context) noexcept {
+    ASSERT_NOTHROW_INVOCABLE(Fn&&);
+    ASSERT_NOTHROW_INVOCABLE(Undo&&);
+    CUcontext previous = nullptr;
+    int changed = 0;
+    CUresult status = enter_context(h_context, &previous, &changed);
+    if (status != CUDA_SUCCESS) {
+        return status;
+    }
+    status = std::invoke(std::forward<Fn>(operation));
+    CUresult composite = exit_context(previous, changed, status);
+    if (status == CUDA_SUCCESS && composite != CUDA_SUCCESS) {
+        bool undo_ok = true;
+        if (undo_requires_target_context) {
+            CUcontext current = nullptr;
+            undo_ok = p_cuCtxGetCurrent(&current) == CUDA_SUCCESS
+                      && current == as_cu(h_context);
+        }
+        if (undo_ok) {
+            std::invoke(std::forward<Undo>(undo));
        }
-        status_ = p_cuCtxSetCurrent(target);
-        changed_ = status_ == CUDA_SUCCESS;
    }
+    return composite;
+}


I played a lot of defense here, and it is easy to see how complex this gets as errors cascade. I partly question the value of this kind of code. Would it be reasonable to call std::abort instead when a key invariant cannot be maintained? If context restoration fails, then whether or not the undo succeeds, the wrong context may remain current, and the program is in big trouble either way.

Andy-Jost · 2026-09-01T22:57:59Z

            event_registry.unregister_handle(b->resource);
            GILReleaseGuard gil;
-            p_cuEventDestroy(b->resource);
+            pw_cuEventDestroy(b->resource);


Replaced deallocation functions (p_*) in destructors with wrapped versions that issue warnings on failure (pw_*).

juenglin · 2026-09-03T22:39:20Z

+def assert_device_operations_use_bound_context(device):
+    """Check that Device operations use its bound context and preserve the ambient context."""
+    bound_context = device.context
+    ambient_context_handle = current_context_handle()


Suggested change

ambient_context_handle = current_context_handle()

ambient_context_handle = current_context_handle()

assert int(bound_context.handle) != ambient_context_handle, (

"Precondition failed: the device's bound context must not be the "

"current (ambient) context."

)

- Fix _SynchronousMemoryResource to record a deallocation token bound to its own context, so Buffer teardown enters the right context regardless of what is current, and works with no context current. Move the class to its own module and resolve the primary context lazily. - LegacyPinnedMemoryResource.device_id returns -1 as documented; texture creation over a pinned buffer works again. - Device.set_current delegates a foreign-device context to its owning device instead of raising, so the save/restore idiom round-trips across devices. - Guard empty context handles once in invoke_in_context(_or_undo); warn when an undo is skipped because the target context is gone. - context_synchronize and context_get_stream_priority_range release the GIL like the other helpers. Rename array/mipmap box accessors to get_box. - Tests: query the driver (cuStreamGetCtx, cross-context cuEventRecord) instead of comparing cached metadata; add sync(), set_current round-trip, pinned texture, and synchronous-resource teardown tests; register device_x2 with the parallel-test plugin. - Move the release note to 1.3.0 and describe sync() as acting on the bound context. Co-Authored-By: Claude Fable 5.1 <[email protected]>

…ce-context

Andy-Jost · 2026-09-04T19:00:32Z

The latest upload addresses Ralf's feedback.

handle_return() takes the whole result tuple, and cuCtxGetDevice() returns a CUdevice that never compares equal to an int; both made the cross-context event-record check raise instead of run (or skip) as intended. Co-Authored-By: Claude Fable 5.1 <[email protected]>

…ce-context

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…text The uniform empty-context guard added for review item 6 broke create_event_handle_noctx, which relied on an empty handle meaning "current context". That helper was itself a NVIDIA#2311-class bug: Stream.wait(stream) and the foreign-array/tensor import paths created their temporary ordering event in whatever context was current, and cuEventRecord rejects an event from a different context than the stream it is recorded on, so cross-device waits failed unless the right device happened to be current. Replace it with create_event_handle_for_stream, which resolves the stream's owning context via cuStreamGetCtx and creates the event there. Callers now check the returned handle and surface the real creation error. With that, every creation helper requires a context and no exception remains. Co-Authored-By: Claude Fable 5.1 <[email protected]>

…back Rebased onto the reviewed head of NVIDIA#2750. Adjustments the rebase needed: - The review's warning for an undo skipped after a failed context restoration is routed through report_cuda_error(), so it carries the CUDA status and becomes a CUDAWarning like every other non-raising report. - invoke_in_context and invoke_in_context_or_undo now reject empty handles themselves, so context_get_device drops its own guard like the other helpers did; enter_context's no-op for empty handles is documented as used only by graph_node_set_params. - _SynchronousMemoryResource moved to its own module; the error-handling test imports it from there. The review's two teardown tests asserted that stderr stayed empty; under the policy a teardown failure is a CUDAWarning, so they assert that no CUDAWarning is issued instead (and are marked thread_unsafe because warning capture is process-global). - report_message() flushes stderr after its last-resort fprintf, so the text is not lost if the process dies right after (review comment). Co-Authored-By: Claude Fable 5.1 <[email protected]>

juenglin · 2026-09-04T22:33:04Z

+                # so pushing/popping a foreign-device context is delegated to
+                # the device that owns it; its own bookkeeping (_context,
+                # _has_inited) is what should track this push, not ours.
+                return Device(ctx._device_id).set_current(ctx)


Worth a sentence in the docstring: delegating here means dev.set_current(foreign_ctx) makes foreign_ctx current on the thread, but does not update dev._has_inited/dev._context -- only Device(ctx._device_id)'s bookkeeping is updated. So immediately after this delegation, dev itself is still considered uninitialized from its own perspective (e.g. dev.create_stream() would still raise "not yet initialized" even though a context is now current on the thread). That's consistent with the comment's reasoning, but it's a non-obvious behavioral subtlety for callers relying on the old set_current contract and would be worth documenting explicitly in the docstring above.

github-actions Bot added the cuda.core Everything related to the cuda.core module label Sep 1, 2026

Andy-Jost force-pushed the ajost/issue-2311-device-context branch 3 times, most recently from a09bebe to 838f850 Compare September 1, 2026 22:12

cuda.core: make Device methods use their bound context

83a3d46

Run context-sensitive Device operations against the Device's bound context while preserving caller state. Centralize context-aware cleanup and synchronous allocation handling so resource lifetimes remain correct.

Andy-Jost force-pushed the ajost/issue-2311-device-context branch from 838f850 to 83a3d46 Compare September 1, 2026 22:26

Andy-Jost commented Sep 1, 2026

View reviewed changes

Andy-Jost requested review from juenglin, leofang, mdboom and rparolin September 1, 2026 23:10

Andy-Jost self-assigned this Sep 2, 2026

Andy-Jost added the bug Something isn't working label Sep 2, 2026

Andy-Jost added this to the cuda.core next milestone Sep 2, 2026

Andy-Jost marked this pull request as ready for review September 2, 2026 00:06

juenglin reviewed Sep 3, 2026

View reviewed changes

Comment thread cuda_core/cuda/core/_memory/_device_memory_resource.pyx

juenglin reviewed Sep 3, 2026

View reviewed changes

Comment thread cuda_core/tests/test_texture_surface.py Outdated

Comment thread cuda_core/tests/test_texture_surface.py Outdated

Andy-Jost and others added 2 commits September 4, 2026 11:17

Merge remote-tracking branch 'origin/main' into ajost/issue-2311-devi…

bc5723c

…ce-context

Andy-Jost requested a review from juenglin September 4, 2026 19:00

Andy-Jost and others added 3 commits September 4, 2026 12:23

Merge remote-tracking branch 'origin/main' into ajost/issue-2311-devi…

ec57bbb

…ce-context

cuda.core: regenerate stub with stubgen-pyx 0.2.22

cdb75ce

Co-Authored-By: Claude Fable 5.1 <[email protected]>

juenglin reviewed Sep 4, 2026

View reviewed changes

juenglin approved these changes Sep 4, 2026

View reviewed changes

Andy-Jost merged commit f84a7cf into NVIDIA:main Sep 4, 2026
209 of 211 checks passed

Navigation Menu

Uh oh!

cuda.core: make Device methods use their bound context - #2750

cuda.core: make Device methods use their bound context#2750
Andy-Jost merged 7 commits into
NVIDIA:mainfrom
Andy-Jost:ajost/issue-2311-device-context

Andy-Jost commented Sep 1, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Sep 1, 2026

Uh oh!

Andy-Jost commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

Andy-Jost commented Sep 1, 2026

Uh oh!

Andy-Jost Sep 1, 2026

Uh oh!

Andy-Jost Sep 1, 2026

Uh oh!

Uh oh!

juenglin Sep 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Andy-Jost commented Sep 4, 2026

Uh oh!

juenglin Sep 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-    ambient_context_handle = current_context_handle()
+    ambient_context_handle = current_context_handle()
+    assert int(bound_context.handle) != ambient_context_handle, (
+        "Precondition failed: the device's bound context must not be the "
+        "current (ambient) context."
+    )

Uh oh!

Conversation

Andy-Jost commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

copy-pr-bot Bot commented Sep 1, 2026

Uh oh!

Andy-Jost commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

Andy-Jost commented Sep 1, 2026

Uh oh!

Andy-Jost Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Andy-Jost Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

juenglin Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Andy-Jost commented Sep 4, 2026

Uh oh!

juenglin Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Andy-Jost commented Sep 1, 2026 •

edited

Loading